Computer vision - Plant seedlings classification¶

Context:¶

Can you differentiate a weed from a crop seedling? The ability to do so effectively can mean better crop yields and better stewardship of the environment.

The Aarhus University Signal Processing group, in collaboration with University of Southern Denmark, has recently released a dataset containing images of unique plants belonging to 12 species at several growth stages

Objective:¶

To implement the techniques learnt as a part of the course.

Learning Outcomes:¶

  • Pre-processing of image data.
  • Visualization of images.•Building CNN.
  • Evaluate the Model.
  • The motive of the project is to make the learners capable to handle images/image classification problems, during this process you should also be capable to handle real image files, not just limited to a numpy array of image pixels.

Import required libraries¶

In [ ]:
# this will help in making the Python code more structured automatically (good coding practice), comment for colab

# %load_ext nb_black 

# library to suppress warnings or deprecation notes
import warnings

warnings.filterwarnings("ignore")

# libraries to help with reading and manipulating data
import pandas as pd
import numpy as np

# library to split data
from sklearn.model_selection import train_test_split, StratifiedKFold

# libaries to help with data visualization
import matplotlib.pyplot as plt
import seaborn as sns

# remove the limit for the number of displayed columns
pd.set_option("display.max_columns", None)

# set the limit for the number of displayed rows
pd.set_option("display.max_rows", 200)

from sklearn import metrics
from sklearn import preprocessing

# library to encoding
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier

# to tune different models
from sklearn.model_selection import train_test_split


# to get diferent metric scores
from sklearn.metrics import (
    recall_score,
    accuracy_score,
    confusion_matrix,
    roc_auc_score,
    roc_curve,
    auc,
    precision_recall_fscore_support
)

#import tenorflow and embedded keras
import tensorflow as tf

import cv2

#sequential api for sequential model 
from tensorflow.keras.models import Sequential, Model

#importing different layers 
from tensorflow.keras.layers import Dense, Dropout, Flatten 
from tensorflow.keras.layers import Conv2D, MaxPooling2D, BatchNormalization, Activation, Input, LeakyReLU,Activation
from tensorflow.keras import backend as K

#to perform one-hot encoding 
from tensorflow.keras.utils import to_categorical 
from tensorflow.keras import losses, optimizers

# regularization method to prevent the overfitting
from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint  

# to apply image transformations
from tensorflow.keras.preprocessing.image  import ImageDataGenerator

Define required functions¶

In [ ]:
# labeled_barplot
def labeled_barplot(data, feature, perc=False, v_ticks=True, n=None):
    """
    Barplot with percentage at the top

    data: dataframe
    feature: dataframe column
    perc: whether to display percentages instead of count (default is False)
    n: displays the top n category levels (default is None, i.e., display all levels)
    """

    total = len(data[feature])  # length of the column
    count = data[feature].nunique()
    if n is None:
        plt.figure(figsize=(count + 1, 5))
    else:
        plt.figure(figsize=(n + 1, 5))

    if v_ticks is True:
        plt.xticks(rotation=90)
        
    ax = sns.countplot(
        data=data,
        x=feature,
        palette="Paired",
        order=data[feature].value_counts().index[:n].sort_values(),
    )

    for p in ax.patches:
        if perc == True:
            label = "{:.1f}%".format(
                100 * p.get_height() / total
            )  # percentage of each class of the category
        else:
            label = p.get_height()  # count of each level of the category

        x = p.get_x() + p.get_width() / 2  # width of the plot
        y = p.get_height()  # height of the plot

        ax.annotate(
            label,
            (x, y),
            ha="center",
            va="center",
            size=12,
            xytext=(0, 5),
            textcoords="offset points",
        )  # annotate the percentage
    plt.show()  # show the plot

# function to load mean image
def find_mean_img(full_mat, title):
    # calculate the average
    mean_img = np.mean(full_mat, axis = 0)
    rgb_weights = [0.2989, 0.5870, 0.1140]
    mean_img = np.dot(mean_img[...,:3], rgb_weights)
    plt.imshow(mean_img/255, cmap=plt.get_cmap('Greys_r'))
    plt.title(f'Average {title}')
    plt.axis('off')
    plt.show()
    return mean_img

Mount google drive¶

In [ ]:
# mount google drive
from google.colab import drive
drive.mount('/content/drive')

Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount("/content/drive", force_remount=True).

Load and understand data¶

In [ ]:
# load image data from .npy
data = np.load("/content/drive/MyDrive/module-9-project/images.npy")

# load labels from csv
labels = pd.read_csv("/content/drive/MyDrive/module-9-project/labels.csv")

Check shape of data¶

In [ ]:
# print the data set information as number of rows and columns
print(f"Shape of image data {data.shape} and number of labels {labels.shape[0]}.")  # f-string

Shape of image data (4750, 128, 128, 3) and number of labels 4750.

Check for NULL Values¶

In [ ]:
# precentage of missing values in columns
round(labels.isna().sum() / labels.isna().count() * 100, 2)
Out[ ]:
Label    0.0
dtype: float64

Observations:

There are no NULL Values

Check for duplicate records¶

In [ ]:
# check duplicate records
labels.duplicated().sum()
Out[ ]:
4738

Observations:

There are 3313 duplicated records. As we know there are various samples for each plant. There is no need to handle the duplicated values.

Describe the data¶

In [ ]:
# let"s view the statistical summary of the data
labels.describe().T
Out[ ]:
count unique top freq
Label 4750 12 Loose Silky-bent 654

Observations:

  1. There are 4750 records.
  2. There are 12 unique data elements.
  3. Loose Silky-bent has most number of records.

Check distinct value counts¶

In [ ]:
# populate value counts
labels.value_counts()
Out[ ]:
Label                    
Loose Silky-bent             654
Common Chickweed             611
Scentless Mayweed            516
Small-flowered Cranesbill    496
Fat Hen                      475
Charlock                     390
Sugar beet                   385
Cleavers                     287
Black-grass                  263
Shepherds Purse              231
Maize                        221
Common wheat                 221
dtype: int64

Exploratory Data Analysis¶

Proportion of categories¶

In [ ]:
# draw a pie chart
plt.figure(figsize=(10, 10))
labels.value_counts().plot(kind='pie')
Out[ ]:
<matplotlib.axes._subplots.AxesSubplot at 0x7f8270655d10>
In [ ]:
# use label_barplot function to plot the graph
labeled_barplot(labels,"Label", True, True)

Observations:

  1. Loose Silky-bent has maximum records.
  2. Followed by Common Chickweed and Scentless Mayweed.
  3. Common Wheat and Maise are the least contributors.
  4. The dataset is imbalanced with one category containing more sample than others.

Plot sample images for each category¶

In [ ]:
# plot sample image of each category

fig = plt.figure(figsize=(10, 10))
for i in range(12):
    if i == 0:
      label = "Small-flowered cranesbill"
      index = 0
    if i == 1:
      label = "Fat Hen"
      index = 496      
    if i == 2:
      label = "Shepherds Purse"
      index = 971    
    if i == 3:
      label = "Common Wheat"
      index = 1202
    if i == 4:
      label = "Common Chickweed"
      index = 1423
    if i == 5:
      label = "Charlock"
      index = 2034
    if i == 6:
      label = "Cleavers"
      index = 2424
    if i == 7:
      label = "Scentless Mayweed"
      index = 2711
    if i == 8:
      label = "Sugar beet"
      index = 3227
    if i == 9:
      label = "Maize"
      index = 3612
    if i == 10:
      label = "Black Grass"
      index = 3833
    if i == 11:
      label = "Loose Silky-bent"
      index = 4096
    ax = fig.add_subplot(4, 3, i+1)

    plt.imshow(data[index], cmap = plt.get_cmap('RdYlGn'))
    plt.title(label)
    plt.axis('off')

plt.show()

Display mean image for each category¶

In [ ]:
# define empty arrays for each category
sfc_data = []
fh_data = []
sp_data = []
cw_data = []
cc_data = []
ch_data = []
cl_data = []
sm_data = []
sb_data = []
m_data = []
bg_data = []
lsb_data = []

# populate the images in each category array
for i in range(4750):
    if i < 496:
      sfc_data.append(data[i])
    if i >= 496 and i < 971:
      fh_data.append(data[i])
    if i >= 971 and i < 1202:
      sp_data.append(data[i])
    if i >= 1202 and i < 1423:
      cw_data.append(data[i])
    if i >= 1423 and i < 2034:
      cc_data.append(data[i])
    if i >= 2034 and i < 2424:
      ch_data.append(data[i])
    if i >= 2424 and i < 2711:
      cl_data.append(data[i])
    if i >= 2711 and i < 3227:
      sm_data.append(data[i])
    if i >= 3227 and i < 3612:
      sb_data.append(data[i])
    if i >= 3612 and i < 3833:
      m_data.append(data[i])
    if i >= 3833 and i < 4096:
      bg_data.append(data[i])
    if i >= 4096:
      lsb_data.append(data[i])


# find means and plot images
sfc_mean = find_mean_img(sfc_data,"Small-flowered cranesbill")
fh_mean = find_mean_img(fh_data, "Fat Hen")
sp_mean = find_mean_img(sp_data, "Shepherds Purse")
cw_mean = find_mean_img(cw_data, "Common Wheat")
cc_mean = find_mean_img(np.array(cc_data), "Common Chickweed")
ch_mean = find_mean_img(np.array(ch_data), "Charlock")
cl_mean = find_mean_img(np.array(cl_data), "Cleavers")
sm_mean = find_mean_img(np.array(sm_data), "Scentless Mayweed")
sb_mean = find_mean_img(np.array(sb_data), "Sugar beet")
m_mean = find_mean_img(np.array(m_data), "Maize")
bg_mean = find_mean_img(np.array(bg_data), "Black Grass")
lsb_mean = find_mean_img(np.array(lsb_data), "Loose Silky-bent")

Observations:

Since there are more than 300 images for each category, the mean images populated doesnt really provide more insight. However from the concentration of the color at specific places indicate that the images for same plant.

Data pre-processing¶

Remove back ground and apply Guassian Blurring¶

In [ ]:
# declare array
gaus_data = []
sets = []; getEx = True

for i in data:
    blurr = cv2.GaussianBlur(i,(5,5),0)
    hsv = cv2.cvtColor(blurr,cv2.COLOR_BGR2HSV)

    # define green parameters
    lower = (25,40,50)
    upper = (75,255,255)
    mask = cv2.inRange(hsv,lower,upper)
    struc = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(11,11))
    mask = cv2.morphologyEx(mask,cv2.MORPH_CLOSE,struc)
    boolean = mask>0
    new = np.zeros_like(i,np.uint8)
    new[boolean] = i[boolean]

    gaus_data.append(new)

    if getEx:
        plt.subplot(2,3,1);plt.imshow(i) # ORIGINAL
        plt.subplot(2,3,2);plt.imshow(blurr) # BLURRED
        plt.subplot(2,3,3);plt.imshow(hsv) # HSV CONVERTED
        plt.subplot(2,3,4);plt.imshow(mask) # MASKED
        plt.subplot(2,3,5);plt.imshow(boolean) # BOOLEAN MASKED
        plt.subplot(2,3,6);plt.imshow(new) # NEW PROCESSED IMAGE
        plt.show()
        getEx = False

# convert the data to array
gaus_data = np.asarray(gaus_data)

Print sample processed images¶

In [ ]:
# visualize the processed image of each category
fig = plt.figure(figsize=(10, 10))

for i in range(12):
    if i == 0:
      label = "Small-flowered cranesbill"
      index = 0
    if i == 1:
      label = "Fat Hen"
      index = 496      
    if i == 2:
      label = "Shepherds Purse"
      index = 971    
    if i == 3:
      label = "Common Wheat"
      index = 1202
    if i == 4:
      label = "Common Chickweed"
      index = 1423
    if i == 5:
      label = "Charlock"
      index = 2034
    if i == 6:
      label = "Cleavers"
      index = 2424
    if i == 7:
      label = "Scentless Mayweed"
      index = 2711
    if i == 8:
      label = "Sugar beet"
      index = 3227
    if i == 9:
      label = "Maize"
      index = 3612
    if i == 10:
      label = "Black Grass"
      index = 3833
    if i == 11:
      label = "Loose Silky-bent"
      index = 4096
    ax = fig.add_subplot(4, 3, i+1)

    plt.imshow(gaus_data[index], cmap = plt.get_cmap('RdYlGn'))
    plt.title(label)
    plt.axis('off')

plt.show()

Observations:

After applying guassian blurring and removing background from images, we are able to remove noise from the images and see the leafs / sapplings clearly.

Onehot encoding of labels¶

In [ ]:
# instantiate labelencoder
labelenc = preprocessing.LabelEncoder()

# encode the labels
labelenc.fit(labels)
encodedlabels = labelenc.transform(labels)

# convert the labesl to categories
clearalllabels = tf.keras.utils.to_categorical(encodedlabels)

Split the data¶

In [ ]:
# normalise the data
gaus_data = gaus_data/255

# splitting data into training and test set of Dependent variable as of original set
X_train,X_test, y_train, y_test = train_test_split(gaus_data,clearalllabels, test_size=0.3, random_state=1)

Rescale and Rotate Images to Prevent overfitting¶

In [ ]:
# instantiate imagedatagenerator to apply rotation and shifts
generator = ImageDataGenerator(rotation_range = 180,zoom_range = 0.1,width_shift_range = 0.1,height_shift_range = 0.1,horizontal_flip = True,vertical_flip = True)

# fit imagedatagenerator
generator.fit(X_train)

Model building¶

MODEL 1 : CNN with Dropout¶

In [ ]:
# select the sequential model
model = tf.keras.models.Sequential()

# input layer
model.add(tf.keras.layers.InputLayer(input_shape=(128,128,3,)))

# here we add a 2D Convolution layer
model.add(tf.keras.layers.Conv2D(64, kernel_size=(3,3), activation='relu'))

# max Pool layer to downsmaple the input representation within the pool_size size
model.add(tf.keras.layers.MaxPool2D(pool_size = (2,2)))

# normalization layer to normalize output using the mean and standard deviation of the current batch of inputs.
model.add(tf.keras.layers.BatchNormalization())

# 2D Convolution layer
model.add(tf.keras.layers.Conv2D(64, kernel_size=(3,3), strides = (1,1), activation='relu'))

# max Pool layer 
model.add(tf.keras.layers.MaxPool2D(pool_size = (2,2)))

# normalization layer
model.add(tf.keras.layers.BatchNormalization())

# 2D Convolution layer
model.add(tf.keras.layers.Conv2D(128, kernel_size=(3,3), strides = (1,1), activation='relu'))

# max Pool layer 
model.add(tf.keras.layers.MaxPool2D(pool_size = (2,2)))

# normalization layer
model.add(tf.keras.layers.BatchNormalization())

# 2D Convolution layer
model.add(tf.keras.layers.Conv2D(128, kernel_size=(3,3), strides = (1,1), activation='relu'))

# max Pool layer 
model.add(tf.keras.layers.MaxPool2D(pool_size = (2,2)))

# global Max Pool layer
model.add(tf.keras.layers.GlobalMaxPool2D())

# flatten the data
model.add(tf.keras.layers.Flatten())

# dense Layers after flattening the data
model.add(tf.keras.layers.Dense(128, activation='relu'))

# dropout to nullify the outputs that are very close to zero and thus can cause overfitting.
model.add(tf.keras.layers.Dropout(0.2))
model.add(tf.keras.layers.Dense(64, activation='relu'))

# normalization layer
model.add(tf.keras.layers.BatchNormalization())

# add Output Layer
model.add(tf.keras.layers.Dense(12, activation='softmax')) # = 12 predicted classes

# compile the model
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

# define call_backs
es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=28)
mc = ModelCheckpoint('best_model.h5', monitor='val_accuracy', mode='max', verbose=1, save_best_only=True)

Print model summary¶

In [ ]:
# print model summary
model.summary()

Model: "sequential_10"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d_59 (Conv2D)          (None, 126, 126, 64)      1792      
                                                                 
 max_pooling2d_58 (MaxPoolin  (None, 63, 63, 64)       0         
 g2D)                                                            
                                                                 
 batch_normalization_40 (Bat  (None, 63, 63, 64)       256       
 chNormalization)                                                
                                                                 
 conv2d_60 (Conv2D)          (None, 61, 61, 64)        36928     
                                                                 
 max_pooling2d_59 (MaxPoolin  (None, 30, 30, 64)       0         
 g2D)                                                            
                                                                 
 batch_normalization_41 (Bat  (None, 30, 30, 64)       256       
 chNormalization)                                                
                                                                 
 conv2d_61 (Conv2D)          (None, 28, 28, 128)       73856     
                                                                 
 max_pooling2d_60 (MaxPoolin  (None, 14, 14, 128)      0         
 g2D)                                                            
                                                                 
 batch_normalization_42 (Bat  (None, 14, 14, 128)      512       
 chNormalization)                                                
                                                                 
 conv2d_62 (Conv2D)          (None, 12, 12, 128)       147584    
                                                                 
 max_pooling2d_61 (MaxPoolin  (None, 6, 6, 128)        0         
 g2D)                                                            
                                                                 
 global_max_pooling2d_10 (Gl  (None, 128)              0         
 obalMaxPooling2D)                                               
                                                                 
 flatten_18 (Flatten)        (None, 128)               0         
                                                                 
 dense_57 (Dense)            (None, 128)               16512     
                                                                 
 dropout_47 (Dropout)        (None, 128)               0         
                                                                 
 dense_58 (Dense)            (None, 64)                8256      
                                                                 
 batch_normalization_43 (Bat  (None, 64)               256       
 chNormalization)                                                
                                                                 
 dense_59 (Dense)            (None, 12)                780       
                                                                 
=================================================================
Total params: 286,988
Trainable params: 286,348
Non-trainable params: 640
_________________________________________________________________

Fit the model¶

In [ ]:
#fit the model on training data
history=model.fit(X_train, 
          y_train,  #It expects integers because of the sparse_categorical_crossentropy loss function
          epochs=20, #number of iterations over the entire dataset to train on
          batch_size=32,
          validation_split=0.20,
          callbacks=[es, mc],
          verbose = 1)#number of samples per gradient update for training

Epoch 1/20
83/84 [============================>.] - ETA: 0s - loss: 2.2902 - accuracy: 0.2300
Epoch 00001: val_accuracy improved from -inf to 0.05113, saving model to best_model.h5
84/84 [==============================] - 6s 62ms/step - loss: 2.2882 - accuracy: 0.2308 - val_loss: 2.5509 - val_accuracy: 0.0511
Epoch 2/20
83/84 [============================>.] - ETA: 0s - loss: 1.5551 - accuracy: 0.4680
Epoch 00002: val_accuracy did not improve from 0.05113
84/84 [==============================] - 5s 58ms/step - loss: 1.5566 - accuracy: 0.4680 - val_loss: 3.1453 - val_accuracy: 0.0511
Epoch 3/20
83/84 [============================>.] - ETA: 0s - loss: 1.2307 - accuracy: 0.5855
Epoch 00003: val_accuracy did not improve from 0.05113
84/84 [==============================] - 5s 55ms/step - loss: 1.2322 - accuracy: 0.5850 - val_loss: 4.3230 - val_accuracy: 0.0511
Epoch 4/20
83/84 [============================>.] - ETA: 0s - loss: 1.0430 - accuracy: 0.6529
Epoch 00004: val_accuracy did not improve from 0.05113
84/84 [==============================] - 5s 59ms/step - loss: 1.0429 - accuracy: 0.6530 - val_loss: 4.7707 - val_accuracy: 0.0511
Epoch 5/20
83/84 [============================>.] - ETA: 0s - loss: 0.8523 - accuracy: 0.7108
Epoch 00005: val_accuracy improved from 0.05113 to 0.06917, saving model to best_model.h5
84/84 [==============================] - 5s 57ms/step - loss: 0.8527 - accuracy: 0.7105 - val_loss: 3.3229 - val_accuracy: 0.0692
Epoch 6/20
83/84 [============================>.] - ETA: 0s - loss: 0.6819 - accuracy: 0.7812
Epoch 00006: val_accuracy did not improve from 0.06917
84/84 [==============================] - 5s 55ms/step - loss: 0.6842 - accuracy: 0.7808 - val_loss: 4.1528 - val_accuracy: 0.0451
Epoch 7/20
83/84 [============================>.] - ETA: 0s - loss: 0.6277 - accuracy: 0.7835
Epoch 00007: val_accuracy improved from 0.06917 to 0.10827, saving model to best_model.h5
84/84 [==============================] - 5s 56ms/step - loss: 0.6275 - accuracy: 0.7838 - val_loss: 3.0730 - val_accuracy: 0.1083
Epoch 8/20
83/84 [============================>.] - ETA: 0s - loss: 0.5381 - accuracy: 0.8223
Epoch 00008: val_accuracy improved from 0.10827 to 0.59398, saving model to best_model.h5
84/84 [==============================] - 5s 56ms/step - loss: 0.5385 - accuracy: 0.8222 - val_loss: 1.2505 - val_accuracy: 0.5940
Epoch 9/20
83/84 [============================>.] - ETA: 0s - loss: 0.4415 - accuracy: 0.8475
Epoch 00009: val_accuracy improved from 0.59398 to 0.79098, saving model to best_model.h5
84/84 [==============================] - 5s 56ms/step - loss: 0.4413 - accuracy: 0.8477 - val_loss: 0.6167 - val_accuracy: 0.7910
Epoch 10/20
83/84 [============================>.] - ETA: 0s - loss: 0.4249 - accuracy: 0.8547
Epoch 00010: val_accuracy improved from 0.79098 to 0.86617, saving model to best_model.h5
84/84 [==============================] - 5s 56ms/step - loss: 0.4267 - accuracy: 0.8545 - val_loss: 0.3819 - val_accuracy: 0.8662
Epoch 11/20
83/84 [============================>.] - ETA: 0s - loss: 0.3295 - accuracy: 0.8908
Epoch 00011: val_accuracy did not improve from 0.86617
84/84 [==============================] - 5s 55ms/step - loss: 0.3314 - accuracy: 0.8902 - val_loss: 0.4822 - val_accuracy: 0.8316
Epoch 12/20
83/84 [============================>.] - ETA: 0s - loss: 0.3385 - accuracy: 0.8844
Epoch 00012: val_accuracy did not improve from 0.86617
84/84 [==============================] - 5s 59ms/step - loss: 0.3384 - accuracy: 0.8842 - val_loss: 0.4269 - val_accuracy: 0.8511
Epoch 13/20
83/84 [============================>.] - ETA: 0s - loss: 0.3127 - accuracy: 0.8968
Epoch 00013: val_accuracy did not improve from 0.86617
84/84 [==============================] - 5s 54ms/step - loss: 0.3159 - accuracy: 0.8962 - val_loss: 0.6690 - val_accuracy: 0.7744
Epoch 14/20
83/84 [============================>.] - ETA: 0s - loss: 0.2857 - accuracy: 0.9021
Epoch 00014: val_accuracy did not improve from 0.86617
84/84 [==============================] - 5s 54ms/step - loss: 0.2853 - accuracy: 0.9023 - val_loss: 0.4014 - val_accuracy: 0.8662
Epoch 15/20
83/84 [============================>.] - ETA: 0s - loss: 0.2317 - accuracy: 0.9213
Epoch 00015: val_accuracy did not improve from 0.86617
84/84 [==============================] - 5s 55ms/step - loss: 0.2330 - accuracy: 0.9211 - val_loss: 0.4339 - val_accuracy: 0.8406
Epoch 16/20
83/84 [============================>.] - ETA: 0s - loss: 0.2266 - accuracy: 0.9281
Epoch 00016: val_accuracy did not improve from 0.86617
84/84 [==============================] - 5s 55ms/step - loss: 0.2264 - accuracy: 0.9282 - val_loss: 0.5923 - val_accuracy: 0.8135
Epoch 17/20
83/84 [============================>.] - ETA: 0s - loss: 0.1461 - accuracy: 0.9559
Epoch 00017: val_accuracy did not improve from 0.86617
84/84 [==============================] - 5s 55ms/step - loss: 0.1482 - accuracy: 0.9549 - val_loss: 1.3906 - val_accuracy: 0.6075
Epoch 18/20
83/84 [============================>.] - ETA: 0s - loss: 0.3261 - accuracy: 0.8859
Epoch 00018: val_accuracy did not improve from 0.86617
84/84 [==============================] - 5s 55ms/step - loss: 0.3287 - accuracy: 0.8853 - val_loss: 0.5121 - val_accuracy: 0.8451
Epoch 19/20
83/84 [============================>.] - ETA: 0s - loss: 0.3006 - accuracy: 0.8976
Epoch 00019: val_accuracy improved from 0.86617 to 0.88271, saving model to best_model.h5
84/84 [==============================] - 5s 56ms/step - loss: 0.3004 - accuracy: 0.8977 - val_loss: 0.3488 - val_accuracy: 0.8827
Epoch 20/20
83/84 [============================>.] - ETA: 0s - loss: 0.1755 - accuracy: 0.9416
Epoch 00020: val_accuracy did not improve from 0.88271
84/84 [==============================] - 5s 55ms/step - loss: 0.1764 - accuracy: 0.9414 - val_loss: 0.8924 - val_accuracy: 0.7398

Observations:

  1. The model tends to overfit on training set but gives lover accuracy on validation set.
  2. Let us try building another model with different set of parameters, which has a different architecture that should generalize well and not overfit.
  3. We can build another model with two Dense Layers having 8 and 16 units respectively.

Model 2: CNN with Dropout after Convolution and having two Dense layers with 16 & 8 units respectively.¶

In [ ]:
# select the sequential model
model2 = tf.keras.models.Sequential()

# input layer
model2.add(tf.keras.layers.InputLayer(input_shape=(128,128,3,)))

# here we add a 2D Convolution layer
model2.add(tf.keras.layers.Conv2D(64, kernel_size=(3,3), activation='relu'))

# max Pool layer to downsmaple the input representation within the pool_size size
model2.add(tf.keras.layers.MaxPool2D(pool_size = (2,2)))

# normalization layer to normalize output using the mean and standard deviation of the current batch of inputs.
model2.add(tf.keras.layers.BatchNormalization())

# 2D Convolution layer
model2.add(tf.keras.layers.Conv2D(64, kernel_size=(3,3), strides = (1,1), activation='relu'))

# max Pool layer 
model2.add(tf.keras.layers.MaxPool2D(pool_size = (2,2)))

# normalization layer
model2.add(tf.keras.layers.BatchNormalization())

# 2D Convolution layer
model2.add(tf.keras.layers.Conv2D(128, kernel_size=(3,3), strides = (1,1), activation='relu'))

# max Pool layer 
model2.add(tf.keras.layers.MaxPool2D(pool_size = (2,2)))

# normalization layer
model2.add(tf.keras.layers.BatchNormalization())

# 2D Convolution layer
model2.add(tf.keras.layers.Conv2D(128, kernel_size=(3,3), strides = (1,1), activation='relu'))

# max Pool layer 
model2.add(tf.keras.layers.MaxPool2D(pool_size = (2,2)))

# global Max Pool layer
model2.add(tf.keras.layers.GlobalMaxPool2D())

# flatten the data
model2.add(tf.keras.layers.Flatten())

# dense Layers after flattening the data
model2.add(tf.keras.layers.Dense(16, activation='relu'))

# dropout to nullify the outputs that are very close to zero and thus can cause overfitting.
model2.add(tf.keras.layers.Dropout(0.2))
model2.add(tf.keras.layers.Dense(8, activation='relu'))

# normalization layer
model2.add(tf.keras.layers.BatchNormalization())

# add Output Layer
model2.add(tf.keras.layers.Dense(12, activation='softmax')) # = 12 predicted classes

# compile the model
model2.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

# define callbacks
es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=20)
mc = ModelCheckpoint('best_model.h5', monitor='val_accuracy', mode='max', verbose=1, save_best_only=True)

Print the model summary¶

In [ ]:
# print model summary
model2.summary()

Model: "sequential_11"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d_63 (Conv2D)          (None, 126, 126, 64)      1792      
                                                                 
 max_pooling2d_62 (MaxPoolin  (None, 63, 63, 64)       0         
 g2D)                                                            
                                                                 
 batch_normalization_44 (Bat  (None, 63, 63, 64)       256       
 chNormalization)                                                
                                                                 
 conv2d_64 (Conv2D)          (None, 61, 61, 64)        36928     
                                                                 
 max_pooling2d_63 (MaxPoolin  (None, 30, 30, 64)       0         
 g2D)                                                            
                                                                 
 batch_normalization_45 (Bat  (None, 30, 30, 64)       256       
 chNormalization)                                                
                                                                 
 conv2d_65 (Conv2D)          (None, 28, 28, 128)       73856     
                                                                 
 max_pooling2d_64 (MaxPoolin  (None, 14, 14, 128)      0         
 g2D)                                                            
                                                                 
 batch_normalization_46 (Bat  (None, 14, 14, 128)      512       
 chNormalization)                                                
                                                                 
 conv2d_66 (Conv2D)          (None, 12, 12, 128)       147584    
                                                                 
 max_pooling2d_65 (MaxPoolin  (None, 6, 6, 128)        0         
 g2D)                                                            
                                                                 
 global_max_pooling2d_11 (Gl  (None, 128)              0         
 obalMaxPooling2D)                                               
                                                                 
 flatten_19 (Flatten)        (None, 128)               0         
                                                                 
 dense_60 (Dense)            (None, 16)                2064      
                                                                 
 dropout_48 (Dropout)        (None, 16)                0         
                                                                 
 dense_61 (Dense)            (None, 8)                 136       
                                                                 
 batch_normalization_47 (Bat  (None, 8)                32        
 chNormalization)                                                
                                                                 
 dense_62 (Dense)            (None, 12)                108       
                                                                 
=================================================================
Total params: 263,524
Trainable params: 262,996
Non-trainable params: 528
_________________________________________________________________

Fit the model¶

In [ ]:
#fit the model on training data
history2=model2.fit(X_train, 
          y_train,  #It expects integers because of the sparse_categorical_crossentropy loss function
          epochs=20, #number of iterations over the entire dataset to train on
          batch_size=32,
          validation_split=0.20,
          callbacks=[es, mc],
          verbose = 1)#number of samples per gradient update for training

Epoch 1/20
83/84 [============================>.] - ETA: 0s - loss: 2.4503 - accuracy: 0.1837
Epoch 00001: val_accuracy improved from -inf to 0.15489, saving model to best_model.h5
84/84 [==============================] - 6s 62ms/step - loss: 2.4504 - accuracy: 0.1835 - val_loss: 2.4785 - val_accuracy: 0.1549
Epoch 2/20
83/84 [============================>.] - ETA: 0s - loss: 2.1762 - accuracy: 0.2880
Epoch 00002: val_accuracy did not improve from 0.15489
84/84 [==============================] - 5s 56ms/step - loss: 2.1765 - accuracy: 0.2880 - val_loss: 2.7889 - val_accuracy: 0.1549
Epoch 3/20
83/84 [============================>.] - ETA: 0s - loss: 1.9578 - accuracy: 0.3445
Epoch 00003: val_accuracy did not improve from 0.15489
84/84 [==============================] - 5s 56ms/step - loss: 1.9580 - accuracy: 0.3444 - val_loss: 2.8041 - val_accuracy: 0.0511
Epoch 4/20
83/84 [============================>.] - ETA: 0s - loss: 1.8444 - accuracy: 0.3855
Epoch 00004: val_accuracy improved from 0.15489 to 0.20000, saving model to best_model.h5
84/84 [==============================] - 5s 56ms/step - loss: 1.8457 - accuracy: 0.3850 - val_loss: 3.2361 - val_accuracy: 0.2000
Epoch 5/20
83/84 [============================>.] - ETA: 0s - loss: 1.7645 - accuracy: 0.4036
Epoch 00005: val_accuracy did not improve from 0.20000
84/84 [==============================] - 5s 59ms/step - loss: 1.7641 - accuracy: 0.4038 - val_loss: 2.6464 - val_accuracy: 0.1684
Epoch 6/20
83/84 [============================>.] - ETA: 0s - loss: 1.6925 - accuracy: 0.4375
Epoch 00006: val_accuracy did not improve from 0.20000
84/84 [==============================] - 5s 55ms/step - loss: 1.6940 - accuracy: 0.4368 - val_loss: 2.9331 - val_accuracy: 0.1714
Epoch 7/20
83/84 [============================>.] - ETA: 0s - loss: 1.5944 - accuracy: 0.4552
Epoch 00007: val_accuracy improved from 0.20000 to 0.22256, saving model to best_model.h5
84/84 [==============================] - 5s 57ms/step - loss: 1.5942 - accuracy: 0.4549 - val_loss: 2.7808 - val_accuracy: 0.2226
Epoch 8/20
83/84 [============================>.] - ETA: 0s - loss: 1.6739 - accuracy: 0.4172
Epoch 00008: val_accuracy improved from 0.22256 to 0.42857, saving model to best_model.h5
84/84 [==============================] - 5s 55ms/step - loss: 1.6737 - accuracy: 0.4173 - val_loss: 1.6198 - val_accuracy: 0.4286
Epoch 9/20
83/84 [============================>.] - ETA: 0s - loss: 1.5263 - accuracy: 0.4857
Epoch 00009: val_accuracy did not improve from 0.42857
84/84 [==============================] - 5s 55ms/step - loss: 1.5265 - accuracy: 0.4857 - val_loss: 1.6897 - val_accuracy: 0.4090
Epoch 10/20
83/84 [============================>.] - ETA: 0s - loss: 1.4122 - accuracy: 0.5218
Epoch 00010: val_accuracy improved from 0.42857 to 0.48571, saving model to best_model.h5
84/84 [==============================] - 5s 56ms/step - loss: 1.4148 - accuracy: 0.5211 - val_loss: 1.5594 - val_accuracy: 0.4857
Epoch 11/20
83/84 [============================>.] - ETA: 0s - loss: 1.3834 - accuracy: 0.5414
Epoch 00011: val_accuracy did not improve from 0.48571
84/84 [==============================] - 5s 55ms/step - loss: 1.3856 - accuracy: 0.5410 - val_loss: 1.9260 - val_accuracy: 0.4060
Epoch 12/20
83/84 [============================>.] - ETA: 0s - loss: 1.6472 - accuracy: 0.4303
Epoch 00012: val_accuracy did not improve from 0.48571
84/84 [==============================] - 5s 59ms/step - loss: 1.6494 - accuracy: 0.4297 - val_loss: 2.1448 - val_accuracy: 0.3970
Epoch 13/20
83/84 [============================>.] - ETA: 0s - loss: 1.5259 - accuracy: 0.4797
Epoch 00013: val_accuracy improved from 0.48571 to 0.61203, saving model to best_model.h5
84/84 [==============================] - 5s 56ms/step - loss: 1.5268 - accuracy: 0.4793 - val_loss: 1.3283 - val_accuracy: 0.6120
Epoch 14/20
83/84 [============================>.] - ETA: 0s - loss: 1.4033 - accuracy: 0.5380
Epoch 00014: val_accuracy did not improve from 0.61203
84/84 [==============================] - 5s 55ms/step - loss: 1.4034 - accuracy: 0.5380 - val_loss: 1.3886 - val_accuracy: 0.5759
Epoch 15/20
83/84 [============================>.] - ETA: 0s - loss: 1.3503 - accuracy: 0.5550
Epoch 00015: val_accuracy improved from 0.61203 to 0.68722, saving model to best_model.h5
84/84 [==============================] - 5s 56ms/step - loss: 1.3511 - accuracy: 0.5549 - val_loss: 1.1283 - val_accuracy: 0.6872
Epoch 16/20
83/84 [============================>.] - ETA: 0s - loss: 1.3187 - accuracy: 0.5697
Epoch 00016: val_accuracy did not improve from 0.68722
84/84 [==============================] - 5s 55ms/step - loss: 1.3215 - accuracy: 0.5688 - val_loss: 1.3708 - val_accuracy: 0.5519
Epoch 17/20
83/84 [============================>.] - ETA: 0s - loss: 1.2922 - accuracy: 0.5730
Epoch 00017: val_accuracy did not improve from 0.68722
84/84 [==============================] - 5s 55ms/step - loss: 1.2930 - accuracy: 0.5722 - val_loss: 1.1696 - val_accuracy: 0.6466
Epoch 18/20
83/84 [============================>.] - ETA: 0s - loss: 1.3868 - accuracy: 0.5365
Epoch 00018: val_accuracy did not improve from 0.68722
84/84 [==============================] - 5s 55ms/step - loss: 1.3889 - accuracy: 0.5361 - val_loss: 1.1465 - val_accuracy: 0.6150
Epoch 19/20
83/84 [============================>.] - ETA: 0s - loss: 1.2971 - accuracy: 0.5651
Epoch 00019: val_accuracy did not improve from 0.68722
84/84 [==============================] - 5s 54ms/step - loss: 1.3002 - accuracy: 0.5647 - val_loss: 1.1520 - val_accuracy: 0.6602
Epoch 20/20
83/84 [============================>.] - ETA: 0s - loss: 1.2899 - accuracy: 0.5587
Epoch 00020: val_accuracy improved from 0.68722 to 0.68872, saving model to best_model.h5
84/84 [==============================] - 5s 57ms/step - loss: 1.2926 - accuracy: 0.5579 - val_loss: 1.0928 - val_accuracy: 0.6887

Observations:

  1. We see that the overfitting has reduced on the training set.
  2. We can build another model with two Dense Layers having 512 and 256 units respectively. And see if the accuract increases.

Model 3: CNN with Dropout after Convolution and having two Dense layers with 512 & 256 units respectively.¶

In [ ]:
# select the sequential model
model3 = tf.keras.models.Sequential()

# input layer
model3.add(tf.keras.layers.InputLayer(input_shape=(128,128,3,)))

# here we add a 2D Convolution layer
model3.add(tf.keras.layers.Conv2D(64, kernel_size=(3,3), activation='relu'))

# max Pool layer to downsmaple the input representation within the pool_size size
model3.add(tf.keras.layers.MaxPool2D(pool_size = (2,2)))

# normalization layer to normalize output using the mean and standard deviation of the current batch of inputs.
model3.add(tf.keras.layers.BatchNormalization())

# 2D Convolution layer
model3.add(tf.keras.layers.Conv2D(64, kernel_size=(3,3), strides = (1,1), activation='relu'))

# max Pool layer 
model3.add(tf.keras.layers.MaxPool2D(pool_size = (2,2)))

# normalization layer
model3.add(tf.keras.layers.BatchNormalization())

# 2D Convolution layer
model3.add(tf.keras.layers.Conv2D(128, kernel_size=(3,3), strides = (1,1), activation='relu'))

# max Pool layer 
model3.add(tf.keras.layers.MaxPool2D(pool_size = (2,2)))

# normalization layer
model3.add(tf.keras.layers.BatchNormalization())

# 2D Convolution layer
model3.add(tf.keras.layers.Conv2D(128, kernel_size=(3,3), strides = (1,1), activation='relu'))

# max Pool layer 
model3.add(tf.keras.layers.MaxPool2D(pool_size = (2,2)))

# global Max Pool layer
model3.add(tf.keras.layers.GlobalMaxPool2D())

# flatten the data
model3.add(tf.keras.layers.Flatten())

# dense Layers after flattening the data
model3.add(tf.keras.layers.Dense(512, activation='relu'))

# dropout to nullify the outputs that are very close to zero and thus can cause overfitting.
model3.add(tf.keras.layers.Dropout(0.2))
model3.add(tf.keras.layers.Dense(256, activation='relu'))

# normalization layer
model3.add(tf.keras.layers.BatchNormalization())

# add Output Layer
model3.add(tf.keras.layers.Dense(12, activation='softmax')) # = 12 predicted classes

# compile the model
model3.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

# define callbacks
es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=20)
mc = ModelCheckpoint('best_model.h5', monitor='val_accuracy', mode='max', verbose=1, save_best_only=True)

Print the model summary¶

In [ ]:
# print model summary
model3.summary()

Model: "sequential_12"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d_67 (Conv2D)          (None, 126, 126, 64)      1792      
                                                                 
 max_pooling2d_66 (MaxPoolin  (None, 63, 63, 64)       0         
 g2D)                                                            
                                                                 
 batch_normalization_48 (Bat  (None, 63, 63, 64)       256       
 chNormalization)                                                
                                                                 
 conv2d_68 (Conv2D)          (None, 61, 61, 64)        36928     
                                                                 
 max_pooling2d_67 (MaxPoolin  (None, 30, 30, 64)       0         
 g2D)                                                            
                                                                 
 batch_normalization_49 (Bat  (None, 30, 30, 64)       256       
 chNormalization)                                                
                                                                 
 conv2d_69 (Conv2D)          (None, 28, 28, 128)       73856     
                                                                 
 max_pooling2d_68 (MaxPoolin  (None, 14, 14, 128)      0         
 g2D)                                                            
                                                                 
 batch_normalization_50 (Bat  (None, 14, 14, 128)      512       
 chNormalization)                                                
                                                                 
 conv2d_70 (Conv2D)          (None, 12, 12, 128)       147584    
                                                                 
 max_pooling2d_69 (MaxPoolin  (None, 6, 6, 128)        0         
 g2D)                                                            
                                                                 
 global_max_pooling2d_12 (Gl  (None, 128)              0         
 obalMaxPooling2D)                                               
                                                                 
 flatten_20 (Flatten)        (None, 128)               0         
                                                                 
 dense_63 (Dense)            (None, 512)               66048     
                                                                 
 dropout_49 (Dropout)        (None, 512)               0         
                                                                 
 dense_64 (Dense)            (None, 256)               131328    
                                                                 
 batch_normalization_51 (Bat  (None, 256)              1024      
 chNormalization)                                                
                                                                 
 dense_65 (Dense)            (None, 12)                3084      
                                                                 
=================================================================
Total params: 462,668
Trainable params: 461,644
Non-trainable params: 1,024
_________________________________________________________________

Fit the model¶

In [ ]:
#fit the model on training data
history3=model3.fit(X_train, 
          y_train,  #It expects integers because of the sparse_categorical_crossentropy loss function
          epochs=20, #number of iterations over the entire dataset to train on
          batch_size=32,
          validation_split=0.20,
          callbacks=[es, mc],
          verbose = 1)#number of samples per gradient update for training

Epoch 1/20
83/84 [============================>.] - ETA: 0s - loss: 1.9022 - accuracy: 0.3614
Epoch 00001: val_accuracy improved from -inf to 0.07519, saving model to best_model.h5
84/84 [==============================] - 6s 61ms/step - loss: 1.9010 - accuracy: 0.3620 - val_loss: 3.9271 - val_accuracy: 0.0752
Epoch 2/20
83/84 [============================>.] - ETA: 0s - loss: 1.1144 - accuracy: 0.6152
Epoch 00002: val_accuracy did not improve from 0.07519
84/84 [==============================] - 5s 55ms/step - loss: 1.1161 - accuracy: 0.6150 - val_loss: 6.3498 - val_accuracy: 0.0316
Epoch 3/20
83/84 [============================>.] - ETA: 0s - loss: 0.8819 - accuracy: 0.7014
Epoch 00003: val_accuracy did not improve from 0.07519
84/84 [==============================] - 5s 55ms/step - loss: 0.8825 - accuracy: 0.7011 - val_loss: 7.4757 - val_accuracy: 0.0331
Epoch 4/20
83/84 [============================>.] - ETA: 0s - loss: 0.7281 - accuracy: 0.7583
Epoch 00004: val_accuracy did not improve from 0.07519
84/84 [==============================] - 5s 55ms/step - loss: 0.7271 - accuracy: 0.7586 - val_loss: 7.5412 - val_accuracy: 0.0511
Epoch 5/20
83/84 [============================>.] - ETA: 0s - loss: 0.4987 - accuracy: 0.8291
Epoch 00005: val_accuracy did not improve from 0.07519
84/84 [==============================] - 5s 55ms/step - loss: 0.5007 - accuracy: 0.8286 - val_loss: 6.4407 - val_accuracy: 0.0737
Epoch 6/20
83/84 [============================>.] - ETA: 0s - loss: 0.4726 - accuracy: 0.8347
Epoch 00006: val_accuracy did not improve from 0.07519
84/84 [==============================] - 5s 55ms/step - loss: 0.4728 - accuracy: 0.8346 - val_loss: 5.7771 - val_accuracy: 0.0316
Epoch 7/20
83/84 [============================>.] - ETA: 0s - loss: 0.4086 - accuracy: 0.8535
Epoch 00007: val_accuracy improved from 0.07519 to 0.47519, saving model to best_model.h5
84/84 [==============================] - 5s 56ms/step - loss: 0.4084 - accuracy: 0.8538 - val_loss: 1.5697 - val_accuracy: 0.4752
Epoch 8/20
83/84 [============================>.] - ETA: 0s - loss: 0.2937 - accuracy: 0.8919
Epoch 00008: val_accuracy improved from 0.47519 to 0.50376, saving model to best_model.h5
84/84 [==============================] - 5s 56ms/step - loss: 0.2935 - accuracy: 0.8921 - val_loss: 1.2711 - val_accuracy: 0.5038
Epoch 9/20
83/84 [============================>.] - ETA: 0s - loss: 0.2993 - accuracy: 0.8976
Epoch 00009: val_accuracy improved from 0.50376 to 0.80451, saving model to best_model.h5
84/84 [==============================] - 5s 56ms/step - loss: 0.2995 - accuracy: 0.8974 - val_loss: 0.5602 - val_accuracy: 0.8045
Epoch 10/20
83/84 [============================>.] - ETA: 0s - loss: 0.2635 - accuracy: 0.9081
Epoch 00010: val_accuracy improved from 0.80451 to 0.84511, saving model to best_model.h5
84/84 [==============================] - 5s 56ms/step - loss: 0.2634 - accuracy: 0.9083 - val_loss: 0.5180 - val_accuracy: 0.8451
Epoch 11/20
83/84 [============================>.] - ETA: 0s - loss: 0.2081 - accuracy: 0.9292
Epoch 00011: val_accuracy did not improve from 0.84511
84/84 [==============================] - 5s 55ms/step - loss: 0.2080 - accuracy: 0.9293 - val_loss: 0.6302 - val_accuracy: 0.8135
Epoch 12/20
83/84 [============================>.] - ETA: 0s - loss: 0.2385 - accuracy: 0.9153
Epoch 00012: val_accuracy did not improve from 0.84511
84/84 [==============================] - 5s 55ms/step - loss: 0.2390 - accuracy: 0.9150 - val_loss: 0.8620 - val_accuracy: 0.7323
Epoch 13/20
84/84 [==============================] - ETA: 0s - loss: 0.3626 - accuracy: 0.8703
Epoch 00013: val_accuracy did not improve from 0.84511
84/84 [==============================] - 5s 55ms/step - loss: 0.3626 - accuracy: 0.8703 - val_loss: 0.9205 - val_accuracy: 0.7549
Epoch 14/20
83/84 [============================>.] - ETA: 0s - loss: 0.2620 - accuracy: 0.9062
Epoch 00014: val_accuracy did not improve from 0.84511
84/84 [==============================] - 5s 55ms/step - loss: 0.2629 - accuracy: 0.9060 - val_loss: 0.6267 - val_accuracy: 0.8165
Epoch 15/20
83/84 [============================>.] - ETA: 0s - loss: 0.1729 - accuracy: 0.9375
Epoch 00015: val_accuracy did not improve from 0.84511
84/84 [==============================] - 5s 55ms/step - loss: 0.1735 - accuracy: 0.9372 - val_loss: 0.9980 - val_accuracy: 0.6692
Epoch 16/20
83/84 [============================>.] - ETA: 0s - loss: 0.1297 - accuracy: 0.9533
Epoch 00016: val_accuracy improved from 0.84511 to 0.85564, saving model to best_model.h5
84/84 [==============================] - 5s 56ms/step - loss: 0.1301 - accuracy: 0.9530 - val_loss: 0.4226 - val_accuracy: 0.8556
Epoch 17/20
83/84 [============================>.] - ETA: 0s - loss: 0.1185 - accuracy: 0.9563
Epoch 00017: val_accuracy improved from 0.85564 to 0.86165, saving model to best_model.h5
84/84 [==============================] - 5s 56ms/step - loss: 0.1184 - accuracy: 0.9564 - val_loss: 0.4437 - val_accuracy: 0.8617
Epoch 18/20
83/84 [============================>.] - ETA: 0s - loss: 0.0664 - accuracy: 0.9793
Epoch 00018: val_accuracy did not improve from 0.86165
84/84 [==============================] - 5s 55ms/step - loss: 0.0683 - accuracy: 0.9786 - val_loss: 0.7385 - val_accuracy: 0.8180
Epoch 19/20
83/84 [============================>.] - ETA: 0s - loss: 0.1543 - accuracy: 0.9469
Epoch 00019: val_accuracy did not improve from 0.86165
84/84 [==============================] - 5s 54ms/step - loss: 0.1548 - accuracy: 0.9466 - val_loss: 0.7660 - val_accuracy: 0.7639
Epoch 20/20
83/84 [============================>.] - ETA: 0s - loss: 0.1167 - accuracy: 0.9608
Epoch 00020: val_accuracy did not improve from 0.86165
84/84 [==============================] - 5s 55ms/step - loss: 0.1166 - accuracy: 0.9609 - val_loss: 0.8366 - val_accuracy: 0.7985

Observations:

  1. We see that model is overfitting again on training set also the accuracy has decreased on validation set.
  2. Let us compare the performance of each model and choose the best fit.

Plot the accuracy of the models¶

Model 1¶

In [ ]:
# summarize history for accuracy
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['training', 'validation'], loc='upper left')
plt.show()

Model 2¶

In [ ]:
# summarize history for accuracy
plt.plot(history2.history['accuracy'])
plt.plot(history2.history['val_accuracy'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['training', 'validation'], loc='upper left')
plt.show()

Model 3¶

In [ ]:
# summarize history for accuracy
plt.plot(history3.history['accuracy'])
plt.plot(history3.history['val_accuracy'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['training', 'validation'], loc='upper left')
plt.show()

Evaluate the model performance on Test Set¶

Model 1¶

In [ ]:
model.evaluate(X_test,np.array(y_test))

45/45 [==============================] - 1s 18ms/step - loss: 0.8811 - accuracy: 0.7235
Out[ ]:
[0.8810718059539795, 0.7235087752342224]

Model 2¶

In [ ]:
model2.evaluate(X_test,np.array(y_test))

45/45 [==============================] - 1s 19ms/step - loss: 1.1579 - accuracy: 0.6477
Out[ ]:
[1.157897710800171, 0.6477193236351013]

Model 3¶

In [ ]:
model3.evaluate(X_test,np.array(y_test))

45/45 [==============================] - 1s 18ms/step - loss: 0.9152 - accuracy: 0.7958
Out[ ]:
[0.9152182936668396, 0.7957894802093506]

Observations:

  1. From the above Model 1 provides best accuracy and Model 3 also does considerably well.
  2. However we have to compare other scores metrics to pick the most suitable model for our analysis and business usage.
  3. Let us also plot the Confusion matrix for Model 1 and Model 3 as they are performing better than Model 2

Draw Confusion Matrix for Model 1 and Model 3¶

Model 1¶

In [ ]:
# define labels
categories = ['Black-grass', 'Charlock', 'Cleavers', 'Common Chickweed', 'Common wheat', 'Fat Hen', 'Loose Silky-bent',
             'Maize', 'Scentless Mayweed', 'Shepherds Purse', 'Small-flowered Cranesbill', 'Sugar beet']

# predictions
y_pred = model.predict(X_test)
y_class = np.argmax(y_pred, axis = 1) 
y_check = np.argmax(y_test, axis = 1) 

# generate confusion matrix
cmatrix = confusion_matrix(y_check, y_class)

# plot heatmap
plt.figure(figsize=(12,10))
sns.heatmap(cmatrix, xticklabels = categories, yticklabels = categories, annot = True, fmt ='g')
Out[ ]:
<matplotlib.axes._subplots.AxesSubplot at 0x7f8273e38f10>

Model 3¶

In [ ]:
# define labels
categories = ['Black-grass', 'Charlock', 'Cleavers', 'Common Chickweed', 'Common wheat', 'Fat Hen', 'Loose Silky-bent',
             'Maize', 'Scentless Mayweed', 'Shepherds Purse', 'Small-flowered Cranesbill', 'Sugar beet']

# predictions
y_pred = model3.predict(X_test)
y_class = np.argmax(y_pred, axis = 1) 
y_check = np.argmax(y_test, axis = 1) 

# generate confusion matrix
cmatrix = confusion_matrix(y_check, y_class)

# plot heatmap
plt.figure(figsize=(12,10))
sns.heatmap(cmatrix, xticklabels = categories, yticklabels = categories, annot = True, fmt ='g')
Out[ ]:
<matplotlib.axes._subplots.AxesSubplot at 0x7f8365998b50>

Observations:

The above two confusion matrices show that the models seem to be working well. Model 1 seem to be the model that we can use for predictions.

Prediction using Model1¶

In [ ]:
# plot one of the images and check the label for the same, we will validate the model outcome using the same
plt.imshow(X_test[1].reshape(128,128,3), cmap='Greys_r')
i=y_test[1]
i=np.argmax(i)
if i == 0:
  label = "Small-flowered cranesbill"
if i == 1:
  label = "Fat Hen"
if i == 2:
  label = "Shepherds Purse"
if i == 3:
  label = "Common Wheat"
if i == 4:
  label = "Common Chickweed"
if i == 5:
  label = "Charlock"
if i == 6:
  label = "Cleavers"
if i == 7:
  label = "Scentless Mayweed"
if i == 8:
  label = "Sugar beet"
if i == 9:
  label = "Maize"
if i == 10:
  label = "Black Grass"
if i == 11:
  label = "Loose Silky-bent"

plt.title(label)

plt.axis('off')
plt.show()
In [ ]:
# predict using model 1 and store the outcome in res
res=model.predict(X_test[1].reshape(1,128,128,3))

i=np.argmax(res)

if i == 0:
  label = "Small-flowered cranesbill"
if i == 1:
  label = "Fat Hen"
if i == 2:
  label = "Shepherds Purse"
if i == 3:
  label = "Common Wheat"
if i == 4:
  label = "Common Chickweed"
if i == 5:
  label = "Charlock"
if i == 6:
  label = "Cleavers"
if i == 7:
  label = "Scentless Mayweed"
if i == 8:
  label = "Sugar beet"
if i == 9:
  label = "Maize"
if i == 10:
  label = "Black Grass"
if i == 11:
  label = "Loose Silky-bent"

print(label)

Sugar beet

Conclusion¶

  1. We have built a Model 1: CNN-model to predict the class of a plant, which works quite well. (Increasing number of epochs and/or adding layers to a model can even increase the performance.
  2. CNN + Maxpooling + Global pooling + Dense is a good combination for image classification.
  3. The model is able to predict the seedling correctly.
In [ ]:
# Module : Introduction to Computer Vision
# Project: Plant seedlings classification
# Submitted by : Ritesh Sharma
# Submission Date :21 Jan 2022